Generated at 2026-02-22T03:33:34.068050+00:00
Overall winner: MiniCluster. MiniCluster won 1 of 5 comparable metrics. Largest deltas: throughput_samples_per_sec (+inf%), latency_p50_ms (+0.00%), latency_p95_ms (+0.00%). No regressions exceeded 5.0%. Consistency findings: none.
| Metric | AutoPerfPy | MiniCluster | Abs Delta | % Delta | Winner |
|---|---|---|---|---|---|
| communication_overhead_percent | N/A | N/A | N/A | N/A | N/A |
| decode_tpt_ms | N/A | N/A | N/A | N/A | N/A |
| energy_per_step_joules | N/A | 3.9279 | N/A | N/A | N/A |
| latency_p50_ms | 0.0000 | 0.0000 | 0.0000 | +0.00% | tie |
| latency_p95_ms | 0.0000 | 0.0000 | 0.0000 | +0.00% | tie |
| latency_p99_ms | 0.0000 | 0.0000 | 0.0000 | +0.00% | tie |
| memory_utilization_percent | 0.0000 | 0.0000 | 0.0000 | +0.00% | tie |
| performance_per_watt | N/A | 8.1625 | N/A | N/A | N/A |
| power_consumption_watts | N/A | 24.8406 | N/A | N/A | N/A |
| scaling_efficiency_pct | N/A | N/A | N/A | N/A | N/A |
| temperature_celsius | N/A | 36.6756 | N/A | N/A | N/A |
| throughput_samples_per_sec | 0.0000 | 202.7609 | 202.7609 | inf | MiniCluster |
| tokens_per_sec | 22.0401 | N/A | N/A | N/A | N/A |
| ttft_ms | N/A | N/A | N/A | N/A | N/A |
Consolidated graph views for quick comparison validation.
Top Normalized Deltas
Positive favors MiniCluster; negative favors AutoPerfPy.
Metric Family Deltas
Family-level signed summary of normalized deltas.
Winner Distribution
Metric-level outcome share across comparable metrics.
Confidence Distribution
Data-strength breakdown for metric conclusions.
Positive values indicate an advantage for MiniCluster; negative values favor AutoPerfPy.
| Metric | Family | Direction | Raw Delta % | Normalized Delta % | Visual | Advantage |
|---|---|---|---|---|---|---|
| latency_p50_ms | latency | low | +0.00% | -0.00% | tie | |
| latency_p95_ms | latency | low | +0.00% | -0.00% | tie | |
| latency_p99_ms | latency | low | +0.00% | -0.00% | tie | |
| memory_utilization_percent | memory | context | +0.00% | +0.00% | context |
Family-level mean of normalized metric deltas (context-only metrics excluded).
| Family | Metrics | Normalized Delta % | Visual | Winner |
|---|---|---|---|---|
| latency | 3 | +0.00% | tie |
Confidence is based on metric availability in both results.
| Metric | Family | AutoPerfPy Available | MiniCluster Available | Direction | Confidence |
|---|---|---|---|---|---|
| communication_overhead_percent | communication | no | no | low | none |
| decode_tpt_ms | other | no | no | low | none |
| energy_per_step_joules | efficiency | no | yes | low | insufficient |
| latency_p50_ms | latency | yes | yes | low | strong |
| latency_p95_ms | latency | yes | yes | low | strong |
| latency_p99_ms | latency | yes | yes | low | strong |
| memory_utilization_percent | memory | yes | yes | context | strong |
| performance_per_watt | performance | no | yes | high | insufficient |
| power_consumption_watts | efficiency | no | yes | low | insufficient |
| scaling_efficiency_pct | other | no | no | high | none |
| temperature_celsius | other | no | yes | high | insufficient |
| throughput_samples_per_sec | performance | yes | yes | high | strong |
| tokens_per_sec | other | yes | no | high | insufficient |
| ttft_ms | other | no | no | low | none |
No consistency regressions detected or all-reduce step data was unavailable.